Image Quality Assessment Algorithms Research Articles

HDRC: a subjective quality assessment database for compressed high dynamic range image

AbstractEvaluating the quality of high dynamic range (HDR) images has emerged as a challenging and contemporary topic with the proliferation of HDR content. In this work, a new HDR compression (HDRC) database is proposed, aiming to provide a benchmark for the development of full-reference HDR image quality assessment (IQA) algorithms when facing the latest HDR compression distortions. In particular, the proposed HDRC database is the first HDR-IQA database to incorporate Versatile Video Coding (VVC) compression distortions, closely associated with practical application scenarios. Furthermore, the proposed HDRC database is currently the largest HDR-IQA database, including 80 reference images and 400 distorted images. Extensive experiments are conducted by studying the performance compared to existing HDR-IQA databases when evaluating three HDR-specific IQA models and nine IQA models prevalent for low dynamic range (LDR) content, revealing the challenges the proposed HDRC database brings. The results indicate that the existing IQA models demonstrate noticeable decreases in accuracy when assessing new compression distortions, underscoring the need for the development of novel HDR-IQA models. Consequently, the suggested HDRC database can serve as a potential database for HDR-IQA research, fostering a comprehensive exploration of the associated fields.

No-Reference Image Quality Assessment Algorithm

The presented article focuses on the development of a local image quality assessment algorithm, specifically designed for analyzing contrast and overall visual data quality. The proposed algorithm aims to enhance the efficiency of image assessment, particularly in conditions of low contrast and the presence of various types of noise. The algorithm's methodology takes into account spectral ranges and provides precise local contrast assessment, making it applicable to a broad spectrum of tasks related to image analysis and enhancement. The developed approach has the potential to improve the quality of visual data by supporting crucial aspects of contrast perception and overall image quality.

McmIQA: Multi-Module Collaborative Model for No-Reference Image Quality Assessment

No reference image quality assessment is a technique that uses computers to simulate the human visual system and automatically evaluate the perceived quality of images. In recent years, with the widespread success of deep learning in the field of computer vision, many end-to-end image quality assessment algorithms based on deep learning have emerged. However, unlike other computer vision tasks that focus on image content, an excellent image quality assessment model should simultaneously consider distortions in the image and comprehensively evaluate their relationships. Motivated by this, we propose a Multi-module Collaborative Model for Image Quality Assessment (McmIQA). The image quality assessment is divided into three subtasks: distortion perception, content recognition, and correlation mapping. And specific modules are constructed for each subtask: the distortion perception module, the content recognition module, and the correlation mapping module. Specifically, we apply two contrastive learning frameworks on two constructed datasets to train the distortion perception module and the content recognition module to extract two types of features from the image. Subsequently, using these extracted features as input, we employ a ranking loss to train the correlation mapping module to predict image quality on image quality assessment datasets. Extensive experiments conducted on seven relevant datasets demonstrated that the proposed method achieves state-of-the-art performance in both synthetic distortion and natural distortion image quality assessment tasks.

An image quality assessment method based on edge extraction and singular value for blurriness

The automatic assessment of perceived image quality is crucial in the field of image processing. To achieve this idea, we propose an image quality assessment (IQA) method for blurriness. The features of gradient and singular value were extracted in this method instead of the single feature in the traditional IQA algorithms. According to the insufficient size of existing public image quality assessment datasets to support deep learning, machine learning was introduced to fuse the features of multiple domains, and a new no-reference (NR) IQA method for blurriness denoted Feature fusion IQA(Ffu-IQA) was proposed. The Ffu-IQA uses a probabilistic model to estimate the probability of each edge detection blur in the image, and then uses machine learning to aggregate the probability information to obtain the edge quality score. After that uses the singular value obtained by singular value decomposition of the image matrix to calculate the singular value score. Finally, machine learning pooling is used to obtain the true quality score. Ffu-IQA achieves PLCC scores of 0.9570 and 0.9616 on CSIQ and TID2013, respectively, and SROCC scores of 0.9380 and 0.9531, which are better than most traditional image quality assessment methods for blurriness.

How subjective CT image quality assessment becomes surprisingly reliable: pairwise comparisons instead of Likert scale.

The aim of this study is to improve the reliability of subjective IQ assessment using a pairwise comparison (PC) method instead of a Likert scale method in abdominal CT scans. Abdominal CT scans (single-center) were retrospectively selected between September 2019 and February 2020 in a prior study. Sample variance in IQ was obtained by adding artificial noise using dedicated reconstruction software, including reconstructions with filtered backprojection and varying iterative reconstruction strengths. Two datasets (each n = 50) were composed with either higher or lower IQ variation with the 25 original scans being part of both datasets. Using in-house developed software, six observers (five radiologists, one resident) rated both datasets via both the PC method (forcing observers to choose preferred scans out of pairs of scans resulting in a ranking) and a 5-point Likert scale. The PC method was optimized using a sorting algorithm to minimize necessary comparisons. The inter- and intraobserver agreements were assessed for both methods with the intraclass correlation coefficient (ICC). Twenty-five patients (mean age 61 years ± 15.5; 56% men) were evaluated. The ICC for interobserver agreement for the high-variation dataset increased from 0.665 (95%CI 0.396-0.814) to 0.785 (95%CI 0.676-0.867) when the PC method was used instead of a Likert scale. For the low-variation dataset, the ICC increased from 0.276 (95%CI 0.034-0.500) to 0.562 (95%CI 0.337-0.729). Intraobserver agreement increased for four out of six observers. The PC method is more reliable for subjective IQ assessment indicated by improved inter- and intraobserver agreement. This study shows that the pairwise comparison method is a more reliable method for subjective image quality assessment. Improved reliability is of key importance for optimization studies, validation of automatic image quality assessment algorithms, and training of AI algorithms. • Subjective assessment of diagnostic image quality via Likert scale has limited reliability. • A pairwise comparison method improves the inter- and intraobserver agreement. • The pairwise comparison method is more reliable for CT optimization studies.

An improved blind/referenceless image spatial quality evaluator algorithm for image quality assessment

An improved blind/referenceless image spatial quality evaluator algorithm for image quality assessment

Dual-stream complex-valued convolutional network for authentic dehazed image quality assessment.

Effectively evaluating the perceptual quality of dehazed images remains an under-explored research issue. In this paper, we propose a no-reference complex-valued convolutional neural network (CV-CNN) model to conduct automatic dehazed image quality evaluation. Specifically, a novel CV-CNN is employed that exploits the advantages of complex-valued representations, achieving better generalization capability on perceptual feature learning than real-valued ones. To learn more discriminative features to analyze the perceptual quality of dehazed images, we design a dual-stream CV-CNN architecture. The dual-stream model comprises a distortion-sensitive stream that operates on the dehazed RGB image, and a haze-aware stream on a novel dark channel difference image. The distortion-sensitive stream accounts for perceptual distortion artifacts, while the haze-aware stream addresses the possible presence of residual haze. Experimental results on three publicly available dehazed image quality assessment (DQA) databases demonstrate the effectiveness and generalization of our proposed CV-CNN DQA model as compared to state-of-the-art no-reference image quality assessment algorithms.

A brain-inspired quality assessment model for sonar image super-resolution

Sonar sensors are vital in the marine industry for detecting underwater targets in challenging conditions. The imaging distance and image resolution are negatively correlated due to the propagation characteristics of sound waves in water. Although Super-Resolution (SR) techniques alleviate this limitation, they introduce complex distortions that may not fit the desired utility of reconstructed sonar images. Quantifying image quality is essential, yet existing Image Quality Assessment (IQA) algorithms fail to simultaneously consider reconstruction distortions and the sonar image task background. Furthermore, the scarcity of sonar images poses challenges for deep-learning-based algorithms. To address these issues, we propose a brain-inspired model for Super-Resolution reconstructed Sonar Image Quality Assessment (SRSIQA) based on transfer learning. On the one hand, we adopt an effective feature extractor to extract multi-level features that align with the human brain recognition process. On the other hand, we develop an SEA-Block to implement feature weight adjustment and multi-level feature scale matching, given that some transferred features do not fit the IQA task well. Experimental results demonstrate the superiority of the proposed method.

Progressive photon mapping algorithm for digital imaging of a space target.

Space target imaging simulation holds great significance for the development of space-based imaging systems and the prediction of imaging tasks. Currently, commonly used rendering algorithms in space target imaging simulation suffer from issues such as radiance calculation results lacking actual physical significance and rendered images containing high-frequency noise. To address these issues, this study proposes a rendering algorithm based on progressive photon mapping in the context of space imaging scenarios, aiming to enhance the accuracy of energy calculations and image rendering for space targets. This algorithm generates multiple photon maps on the target surface through multiple iterations, retrieves photon information near the observation point based on these photon maps, and thus obtains the radiance of the observation direction vector. This study evaluates the quality of rendered images using no-reference image quality assessment algorithms. The results demonstrate that this algorithm can enhance image rendering quality in specific imaging scenarios, consequently improving the accuracy of space target recognition. By comparing the calculated values of this algorithm with the theoretical radiance values for diffuse material, the accuracy of the radiance calculation results of this algorithm is verified, which can provide significant reference values for the selection of backend detectors.

Learning Deep Blind Quality Assessment for Cartoon Images.

Although the cartoon industry has developed rapidly in recent years, few studies pay special attention to cartoon image quality assessment (IQA). Unfortunately, applying blind natural IQA algorithms directly to cartoons often leads to inconsistent results with subjective visual perception. Hence, this brief proposes a blind cartoon IQA method based on convolutional neural networks (CNNs). Note that training a robust CNN depends on manually labeled training sets. However, for a large number of cartoon images, it is very time-consuming and costly to manually generate enough mean opinion scores (MOSs). Therefore, this brief first proposes a full reference (FR) cartoon IQA metric based on cartoon-texture decomposition and then uses the estimated FR index to guide the no-reference IQA network. Moreover, in order to improve the robustness of the proposed network, a large-scale dataset is established in the training stage, and a stochastic degradation strategy is presented, which randomly implements different degradations with random parameters. Experimental results on both synthetic and real-world cartoon image datasets demonstrate the effectiveness and robustness of the proposed method.

High Dynamic Range Image Quality Assessment Based on Frequency Disparity

In this paper, a novel and effective image quality assessment (IQA) algorithm based on frequency disparity for high dynamic range (HDR) images is proposed, termed as local-global frequency feature-based model (LGFM). Motivated by the assumption that the human visual system (HVS) is highly adapted for extracting structural information and partial frequencies when perceiving the visual scene, the Gabor and the Butterworth filters are applied to the luminance component of the HDR image to extract the local and global frequency features, respectively. The similarity measurement and feature pooling strategy are sequentially performed on the frequency features to obtain the predicted single quality score. The experiments evaluated on four widely used benchmarks demonstrate that the proposed LGFM can provide a higher consistency with the subjective perception compared with the state-of-the-art HDR IQA methods. Our code is available at: https://github.com/eezkni/LGFM.

Full-Reference Image Quality Assessment Based on Multi-Channel Visual Information Fusion

This study focuses on improving the objective alignment of image quality assessment (IQA) algorithms with human visual perception. Existing methodologies, predominantly those based on the Laplacian of Gaussian (LoG) filter, often neglect the impact of color channels on human visual perception. Consequently, we propose a full-reference IQA method that integrates multi-channel visual information in color images. The methodology begins with converting red, green, blue (RGB) images into the luminance (L), red–green opponent color channel (M), blue–yellow opponent color channel (N) or LMN color space. Subsequently, the LoG filter is separately applied to the L, M, and N channels. The convoluted components are then fused to generate a contrast similarity map using the root-mean-square method, while the chromaticity similarity map is derived from the color channels. Finally, multi-channel LoG filtering, contrast, and chromaticity image features are connected. The standard deviation method is then used for sum pooling to create a full-reference IQA computational method. To validate the proposed method, distorted images from four widely used image databases were tested. The evaluation, based on four criteria, focused on the method’s prediction accuracy, computational complexity, and generalizability. The Pearson linear correlation coefficient (PLCC) values, recorded from the databases, ranged from 0.8822 (TID2013) to 0.9754 (LIVE). Similarly, the Spearman rank-order correlation coefficient (SROCC) values spanned from 0.8606 (TID2013) to 0.9798 (LIVE). In comparison to existing methods, the proposed IQA method exhibited superior visual correlation prediction accuracy, indicating its promising potential in the field of image quality assessment.

An Optimization-Based Family of Predictive, Fusion-Based Models for Full-Reference Image Quality Assessment.

Given the reference (distortion-free) image, full-reference image quality assessment (FR-IQA) algorithms seek to assess the perceptual quality of the test image. Over the years, many effective, hand-crafted FR-IQA metrics have been proposed in the literature. In this work, we present a novel framework for FR-IQA that combines multiple metrics and tries to leverage the strength of each by formulating FR-IQA as an optimization problem. Following the idea of other fusion-based metrics, the perceptual quality of a test image is defined as the weighted product of several already existing, hand-crafted FR-IQA metrics. Unlike other methods, the weights are determined in an optimization-based framework and the objective function is defined to maximize the correlation and minimize the root mean square error between the predicted and ground-truth quality scores. The obtained metrics are evaluated on four popular benchmark IQA databases and compared to the state of the art. This comparison has revealed that the compiled fusion-based metrics are able to outperform other competing algorithms, including deep learning-based ones.

Objective Prediction of Human Visual Acuity Using Image Quality Metrics

This work addresses the objective prediction of human uncorrected decimal visual acuity, an unsolved challenge due to the contribution of both physical and neural factors. An alternative approach to assess the image quality of the human visual system can be addressed from the image and video processing perspective. Human tolerance to image degradation is quantified by mean opinion scores, and several image quality assessment algorithms are used to maintain, control, and improve the quality of processed images. The aberration map of the eye is used to obtain the degraded theoretical image from a set of natural images. The amount of distortion added by the eye to the natural image was quantified using different image processing metrics, and the correlation between the result of each metric and subjective visual acuity was assessed. The correlation obtained for a model based on a linear combination of the normalized mean square error metric and the feature similarity index metric was very good. It was concluded that the proposed method could be an objective way to determine subjects’ monocular and uncorrected decimal visual acuity with low uncertainty.

PGF-BIQA: Blind image quality assessment via probability multi-grained cascade forest

Blind image quality assessment (BIQA) aims to automatically predict the perceptual quality of a digital image without accessing its pristine reference and plays an important role in computer vision and image analysis. In fact, the difference in the subjective perception of image quality by evaluators greatly affected quality scores of the images. Aiming at the problem that the existing models cannot take into account the subjective perception differences in people, a blind image quality assessment algorithm via probability gcForest (PGF-BIQA) is proposed. Specifically, we use five qualitative labels to replace the specific quality score and extract image color and texture features to represent image quality. Unlike the widely used neural network, gcForest is ensemble by decision trees which randomly extract different features of various modes to learn the relationship between label and quality features. Therefore, a probability gcForest is designed, combining the frequency probability model and the gcForest, using probability confidence to represent the multiple decision modes results, and describing the subjective perception differences in evaluators. Next, through quality anchors and probability confidences of different classes, PGF-BIQA achieves image quality assessment. Moreover, this method proposes an image classification method that resolves the problem of an unbalanced number of images of different classes and the calculation process of the algorithm is simple and effective. Extensive experiments demonstrate that the proposed method is highly consistent with human perception and outperforms many state-of-the-art BIQA algorithms.

Research on No-reference Image Quality Assessment Algorithm Based on Generative Adversarial Networks

Currently, with the massive generation and transmission of digital images, especially the rapid development of the Internet and mobile Internet industries where images are widely used, the study of reference-free image quality assessment has attracted much attention in academia and practical applications, and is a popular research direction in the field of computer vision. In response to the low performance of existing reference-free image quality assessment algorithms in the face of real distorted images, a reference-free image quality assessment algorithm based on generative adversarial networks is proposed. Firstly, the generator structure is changed, the U-Net structure is improved, and the channel attention mechanism SeNet structure is introduced to update the feature map after down sampling. Secondly, a feature similarity measurement system is incorporated and a dual discriminator structure is used to discriminate multiple groups of images. The FPN structure is combined in the feature extractor to produce a multi-scale feature representation. Experiments are conducted on KonIQ-10k dataset and LIVEC dataset, and the experimental results show that the algorithm exhibits good prediction accuracy as well as good generalization performance in the face of real distorted images.

No-Reference Image Quality Assessment Using the Statistics of Global and Local Image Features

Methods of image quality assessment are widely used for ranking computer vision algorithms or controlling the perceptual quality of video and streaming applications. The ever-increasing number of digital images has encouraged the research in this field at an accelerated pace in recent decades. After the appearance of convolutional neural networks, many researchers have paid attention to different deep architectures to devise no-reference image quality assessment algorithms. However, many systems still rely on handcrafted features to ensure interpretability and restrict the consumption of resources. In this study, our efforts are focused on creating a quality-aware feature vector containing information about both global and local image features. Specifically, the research results of visual physiology indicate that the human visual system first quickly and automatically creates a global perception before gradually focusing on certain local areas to judge the quality of an image. Specifically, a broad spectrum of statistics extracted from global and local image features is utilized to represent the quality-aware aspects of a digital image from various points of view. The experimental results demonstrate that our method’s predicted quality ratings relate strongly with the subjective quality ratings. In particular, the introduced algorithm was compared with 16 other well-known advanced methods and outperformed them by a large margin on 9 accepted benchmark datasets in the literature: CLIVE, KonIQ-10k, SPAQ, BIQ2021, TID2008, TID2013, MDID, KADID-10k, and GFIQA-20k, which are considered de facto standards and generally accepted in image quality assessment.

No-reference Quality Assessment for Contrast-distorted Images Based on Gray and Color-gray-difference Space

No-reference image quality assessment is a basic and challenging problem in the field of image processing. Among them, contrast distortion has a great impact on the perception of image quality. However, there are relatively few studies on no-reference quality assessment of contrast-distorted images. This article proposes a no-reference quality assessment algorithm for contrast-distorted images based on gray and color-gray-difference (CGD) space. In terms of gray space, we consider the local and global aspects, and use the distribution characteristics of the grayscale histogram to represent global features, while local features are described by the fusion of Local Binary Pattern (LBP) operator and gradient. In terms of CGD space, we first randomly extract patches from the entire image and then extract appropriate quality perception features in the patch’s CGD histogram. Finally, the AdaBoosting back propagation (BP) neural network is used to train the prediction model to predict the quality of the contrast-distorted image. Extensive analysis and cross-validation are carried out on five contrast-related image databases, and the experimental results have proved the superiority of this method compared with recent related algorithms.

A large-scale image database for benchmarking mobile camera quality and NR-IQA algorithms

Camera-oriented quality assessment (CQA) differs from traditional image quality assessment (IQA) in that “distorted” images are straight out of real devices instead of various types and levels of artificial operations. However, despite its value for both customers and manufacturers, academic and industrial fields, there are few CQA databases created years ago. To reflect recent mobile imaging advancements, we present a new massive Phone Camera Benchmarking database (PCB2021) in this paper. In PCB2021, 40 modern phone units featuring photography are simultaneously compared in 182 scenes for a total of 7280 images, which can be classified into six categories (sub-datasets) based on different focal lengths and user cases: main camera (107), ultra-wide (20), 2×, 3×, 5× zoom lenses (26, 8, 10) and night-mode (11). The shooting process begins from 7:00 am to 11:00 pm and lasts more than a month. In the subjective study, to overcome the high-resolution-induced overall quality evaluation difficulty, five image attributes: exposure/contrast, color, sharpness, graininess, artifacts are assessed separately on each dataset. To reduce ranking complexity for large-scale cameras, a dynamic anchor ruler method is proposed to obtain quality orders efficiently. With the constructed PCB2021, we further evaluate 15 mainstream no-reference (NR) IQA algorithms. The finding is that for zoomed images, sharpness metrics can achieve Spearman correlation coefficients above 0.8, while for the subtle main camera and night-mode images, performances of all fifteen algorithms drop down quickly, i.e. 0.1-0.2 for the former and ∼0.5 for the latter. The entire database, expert rankings and algorithm performance reports will be freely available on request.

Image Quality Assessment for Magnetic Resonance Imaging

Image quality assessment (IQA) algorithms aim to reproduce the human's perception of the image quality. The growing popularity of image enhancement, generation, and recovery models instigated the development of many methods to assess their performance. However, most IQA solutions are designed to predict image quality in the general domain, with the applicability to specific areas, such as medical imaging, remaining questionable. Moreover, the selection of these IQA metrics for a specific task typically involves intentionally induced distortions, such as manually added noise or artificial blurring; yet, the chosen metrics are then used to judge the output of real-life computer vision models. In this work, we aspire to fill these gaps by carrying out the most extensive IQA evaluation study for Magnetic Resonance Imaging (MRI) to date (14,700 subjective scores). We use outputs of neural network models trained to solve problems relevant to MRI, including image reconstruction in the scan acceleration, motion correction, and denoising. Our emphasis is on reflecting the radiologist's perception of the reconstructed images, gauging the most diagnostically influential criteria for the quality of MRI scans: signal-to-noise ratio, contrast-to-noise ratio, and the presence of artifacts. Seven trained radiologists assess these distorted images, with their verdicts then correlated with 35 different image quality metrics (full-reference, no-reference, and distribution-based metrics considered). The top performers -- DISTS, HaarPSI, VSI, and FID-VGG16 -- are found to be efficient across three proposed quality criteria, for all considered anatomies and the target tasks.